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ABSTRACT 


A statistical analysis was conducted to investigate and 
evaluate the relationship of English comprehension levels 
of foreign students with their graduate academic performances 
at the Naval Postgraduate School. 

In this study, English comprehension level test scores 
of foreign students were correlated with the final academic 
grades received at the school. 

‘The results obtained revealed a positive, but marginally 
useful, relationship between the two variables. The results 
were also analyzed by curricula subgroups and by military 
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I. INTRODUCTION 


A. GENERAL 

Many factors determine the academic achievement of a 
Student. if the learning situation is in*a foreign country, 
adequate communication capability would be a most critical 
and basic requirement for successful performance by foreign 
students. 

This study was intended to investigate and evaluate how 
the English language comprehension levels of foreign stu- 
dents from countries where English is not the principal 
language are related to their academic performance at the 
Naval Postgraduate School (NPS). 

This study will also provide an occasion to examine the 
effectiveness of a current English comprehension level test 
in predicting academic grades in the postgraduate study at 


the NPS, 


be ~WEPFINITIONS 

1. Foreign Student - A person from a country where 
English is not the principal language and who is required 
tOmtdakema English ecomprehensa cnellevel testeto study at NPo- 

4, ObeBiCy Wem eRe TON , elhelWayehio mcs (oad ake WU 
academic grade average computed from the numerical standards 
established by the NPSwe (See Chapter Ili. 8) 

See ciiclash Gonprenens son. Level a ECL) eA numerical 


Gerademwol Lnelisinweroticiemey converted to qualification 





levels, as established in a numerical system by the Defense 
Language Institute (DLI) to show the English language 
capability of non-English speaking nationals. (See Chapter 


Hit Bs) 


C. VALUE OF THE STUDY 

All foreign students are required to have a certain 
English language capability in order to be admitted to the 
schools in the United States, but the facility with which 
one communicates in a foreign language rarely equals that 
in his native tongue. Hence, one can expect English 
language deficiencies to be a problem among foreign students. 

Without a sincere understanding of the language problen, 
there might exist not only an unintentional collision of 
feelings, but also discrimination in evaluating students. 
Grades may sometimes be inflated due to the tendency of 
professors to be quite lenient in evaluating foreign stu- 
dents, or sometimes grades may be deflated because the 
same norms may be used for all students. 

The findings of this study may suggest some ideas for 
Benoo! sauchoritic€S sm sScteine Up pollelesmeoncarning ror. 
eenestudents, and for the@agenciics responsible; for the 
English proficiency test to review the measures of®Eng] iicom 
enicime sor efore in) students sadmit Ledeen Vari OUssleVelLsnen 
Schools) in the United States. 

Further, this study may help individual foreign coun- 
irucs examine their scréenaing procedures for their cairli - 


dateseto enter NPS. 
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This study will also provide some basis for further 
study in determining factors predictive of academic achieve- 


ment of non-English speaking nationals at the NPS. 
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ieee A LEMEN TOO THEO ROBLEM 


A. THE PROBLEM 

Because of a lack of proficiency in English, many for- 
eign students in the United States have difficulty in making 
a satisfactory adjustment to not only their academic work 
but also to their daily associates. It is hardly an exag- 
geration to say that during the foreign students' sojorn 
in the United States, everything hinges on his ability to 
communicate adequately -- with his teachers, his books, 
his fellow students, and his associates in daily life. 
Most) of what he learns must be filtered through @ communi- 
cation process, and good communication provides the setting 
in which other problems of adjustment are most easily 
solved, while blocked or distorted communication can give 
rise to a vicious spiral of other personel difficuities. 

Edward Cieslals stated, in his study of Foreign Student 
in American Colleges, that of a dozen qualifications which 
collegiate institutions consider desirable for the admis- 
sion of foreign students, sufficient mastery of English to 
name einem tom carryead fuli= programmes rated most 
important [Ref. l]. 

Pvc iamtnent CTebe reise Std ain I9Oomeeterminedm@nae 
53 percent of the military students programmed annually for 
technical/professional training in the U.S. service schools 


nad English comprehension below the level required for 


successful understanding of course offerings. This did 


eZ 





not mean that the students were necessarily given failing 
grades [Ref. 2]. 

The brief discussions which have been presented thus 
far give rise to a number of questions such as, How can one 
measure or determine adequacy of English capability for 
foreign students being educated in the United States?, To 
what degree is a student's success in a given course based 
on his comprehension of English?, Can a student's level of 
English comprehension provide some basis for predicting his 
success in a formal training course?, Are the objectives of 
foreign training being achieved? 

This thesis is concerned with the problem of investi- 
gating and evaluating the relationship of the English com- 
prehension level of foreign students with their academic 
performance at the Naval Postgraduate School. 

If a relationship between English proficiency and achieve- 
ment in courses were determined, it would be possible to pro- 
vide a means for screening candidates for study and predicting 


their academic success at NPS, 


B. ASSUMPTIONS AND LIMITATIONS 

Some assumptions and limitations were taken into account 
during inquiring into the problem. These assumptions and 
limitations are as follows: 

(ieee SeOst Ofethescunriculla at NPS ane 18 months or 
longer in length, foreign students quite possibly increase 
their English profictency during their time in the United 


otates. The degree of improvement may be different depending 
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Upeneci induyidual s driligence, Untelligence, persondlity, 
motivation, etc., and the additional language proficiency 
could affect his academic achievement. It was assumed, 
however, that the original ECL score of a foreign student 
acquired prior to entry into NPS might be a valid predictor 
of his NPS performance. 

2. Many factors which could affect a foreign student's 
academic success at NPS, such as individual background, 
family status, ethnic and national characteristics, economic 
and social status, and motivation, are ignored, and only 
Engi Simpronrerency aS represented by ECL scores was re- 
lated to academic performance in this study. 

3. The prevailing system of assigning alphabetical 
course grades probably does not effectively discriminate 
between students whose performance is not the same. This 
is because identical grades may be assigned to students even 
though their performance covers a relatively wide range. 
Fluctuations of grading policies from one class to another, 
and from one professor to another, also tend to distort the 
validity and reliability of grades. 

4. Because of admission policies, foreign students 
commmemcomthe Neo all had ECliescones of 80 or higher, the 
frequency distribution of the test scores (ECL) was skewed. 
This restriction of range tends to lower the correlations 
between ECL and other variables [Ref. 3]. 

>. voampling for this research was limited to only the 


recent two years of graduation classes as descriptions for 


14 


personal data preceding those classes were not available. 
This limited sampling may not be representative of the 
whole universe of foreign students able to come to NPS. 

6. It was hypothesized that the relationship of English 
comprehension to academic grades would be stronger in the 
curricula having many verval or administrative type courses 
than in the curricula agyiiscian.s very heavily on mathematics 
or engineering type courses. This hypothesis was examined 
by dividing the sample into engineering and non-engineering 
curricula groups. 

7. It was also decided to investigate whether any dif- 
ferences existed between student's military rank. This was 
examined by comparing the statistical results of two rank 
groups, one with ranks of Lieutenant and below and another 


with ranks of Lieutenant Commander and above. 


Ge eRELATED STUDIES PERTINENT TO THE PROBLEM 

1. The Army Study - A George Washington University 
Study by Dr. George H. Brown (the Brown Study) was concerned 
with foreign students in U. S. Army schools [Ref. 4]. The 
objectives of the research were to assess the academic 
achievement of foreign students in four selected U.S. Army 
schools and to investigate the relationship between English 
language proficiency and academic achievement. The sample 
micinided “i il59 foreipon Studentsmrepresenting 49 countriles 
throughout the world. 

ise S Cideeenc vCa kode Liate Umpc racniteah the torergn 


students were seriously handicapped by English language 
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deficiency, and if standards for United States students had 
been rigidly enforced, 22 percent of all courses completed 
would have been failed: included in the failures would have 
been 51 percent of the students who had poor English pro- 
ficiency and 15 percent of the students who had excellent 
English proficiency as judged by the instructors. 

Like @repor concluded that the actual educational or 
training objective of the foreign student training program 
was not being met, and that English language deficiency was 
a significant factor in the underachievement of foreign 
students. 

The findings and conclusions of the Brown study were 
later strongly supported by a Department of Defense Study 
of English Language Training of Foreign Students [Ref. 5]. 

Zeaeenca Airy Worces study - Ralph ©. Emdninnteen conducted 
a study on English comprehension level as a predictor of 
achievement for foreign military in training in the Under- 
Si uare Pilot Training Coursie of thesUnited States Air 
Force [Ref. 6]. He-inves tigated the relationship of 
LiGausnEpEaOrLTGlencyamas measured by ECL, with the academic 
and flying training achievements of foreign students. 

The sample included 169 foreign student pilots trained 
mm the Undéreraduate Pilot Training ~T-28 Course from 
January 1965 through September 1966. 

The study did not accept the following hypothesis be- 
Cause Sonetheskow correlation: ECL may be usied to predict 
a foreign student's technical flying training or his final 


flying training grade. However, another hypothesis, ECL 
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may be used to predict a foreign student's final academic 
grade, was accepted, based upon a correlation coefficient 
of .7425, which was sufficiently high to provide for mean- 
ingful prediction of academic grades from ECL scores. 

The study concluded that proficiency in the English 
language, as measured by the ECL, may be considered as an 
PIPSmUlewecONtrmeucIng tactor am Cher prediction of agademic 
success in Undergraduate Pilot Training. 

Sree oOthereoocaal Studies . Eeeides EhemECEL deseribed 
previously, several tests of English ability have been con- 
structed specifically for use with foreign students enter- 
ing university training. Among these tests are the Test 
of English as a Foreign Language (TOEFL), the University 
of Michigan Tests (the Lado Test) developed at the English 
Language Institute at Michigan, and the Pennsylvania State 
University English Language Proficiency Test. 

These tests, as predictors of success for foreign stu- 
dents, were evaluated by Chase and Stallings in an Indiana 
University Monograph. Their data were from 526 foreign 
students [Ref. 7]. 

Tieyvereuind siatmene LORFL and the Pennsylvania State 
English Language Prorretcmey, lest Weregm@a,sionitficantly 
correlated egy Prade Waitt avetavem (Gurnee COrre lation 
COecrtmucients From) .1963 to .2273 were found to be signifi- 
Gdtmimetne s1eScarchOuLethecse weorrelations were evaluated 
as too low to have practical effectiveness. 

Chase and Stallings concluded that-the tests predicted 


achievement in only a casual way, and they suggested that 


ey 





there is little prospect for relating scores on tests in 
English with a subsequent GPA in college. 

Walter P. Allen investigated the relationship of Lado 
test scores with first semester grades of ninety students 
at the University of Houston [Ref. 7]. This study showed 
product-moment correlations coefficients between .21 and 
.32, and the study indicated that there is a slight cor- 
relation between the English test scores and grades made 
at the University. The correlations were especially marked 
when only English grades were considered. 

In summation, all the above studies showed a positive 
correlation between a foreign student's English facility 
and his academic adjustment, but the results of these studies 
were not conclusive. They showed that the correlation of 
English proficiency to academic achievement of students at 
military schools was stronger than at universities and 


colleges. 
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III. EXPERIMENTAL DESIGN AND PROCEDURES 


A. POPULATION AND SAMPLE 
1. Population 
The population for this study was considered to be 
of infinite or indeterminate size, including all foreign 
students who had entered the graduate courses of NPS in the 
past, or who would enter the NPS in the future. The size 
of the population largely depends upon the quota available 
at the NPS and the policies of the United States and the 
individual foreign countries. 
Z. sample 
Table I shows the number of and percent of foreign 
students graduated from NPS during the period of academic 
year (AY) 1973 and 1974, which depicts approximately 10 to 


15>percent of foreign students in each class of the year. 


Table I. Number and percent of foreign students graduated 
lened i Caneel eased GAY S107 drs 

Graduate Total Number LW cis ¢ Foreign Students 
Class of Graduates Students Number Percent 

AY-1973 7 641 69 Ory 
QTR-I V2 106 16 Sal 
QTR-II 189 164 25 5 aA 
QTR-III 169 165 4 2.4 
TRS ILA 230 206 24 ery, 

AY-1974 543 469 74 Seo 
Omen. 149 VA 25 IRS 
© Tie 108 88 20 eh ed 
QTR-III 125 116 9 foots 
OT Rey 161 139 jis leet 

Ota 25 1,110 143 1 ae, 


*Compiled from the roster filled in the Registrar's Office, 
Naval Postgraduate School. 
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The eight consecutive NPS graduate classes of lst 
Quarter, AY-1973 through 4th Quarter, AY-1974, which cor- 
respond to graduation classes of September 1972 through 
July 1974, constituted the sample representing the indeter- 
minate population of all foreign students. 

Purposive sampling was used because transcripts of 
foreign students having ECL scores recorded on them were not 
available for graduate classes preceding September 1972. 

The sample included 110 foreign students represent- 
ing 17 countries throughout the world as shown in Table II. 
This was the maximum size of sample available at the present 
time, as there were some students from countries where the 
ECL test was not required, or whose transcripts were missing. 


Table II. Number and source of foreign students included 
in the sample: by country 


Country No. Students Country No. Students 
Argentina 1 Korea 10 
Brazil 6 Peau 10 
China 5 Portugal S 
Chile 4 Thailand 15 
Ecuador 1 fie key, 14 
arcece 8 Urguay 

Indonesia i Vente ame | a 

Iran yd Vietnam ly 
Japan 1 Total 110 


Micmantlisn PprOLtctTechHeleS ol toreion students as 
measured by ECL scores were obtained from records on file 


at the? NPS Mrlitary Personnel Office, and the final average 
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Table III. Number of foreign students included in the 
sample: by ranks* 


LT LCDR CDR CAPTAIN TOTAL 


56 36 15 3 110 


*Rank shown is the rank held upon reporting to the 
Naval Postgraduate School 


*Ranks of other services than Navy are converted to 
the Navy equivalent 
Table IV shows the constitution of the sample by 


academic year and curriculum. 


Table IV. Constitution of the sample by class and curricula. 


No. Students 
Graduated in 


Department Curr letica AY-1973 AY-1974 Total 
Operations Operations ) 
Research §& Research 14 a4 38 
Administra- 
——_ Management 9 5 14 
Sciences Computer 7 1 3 
science 
Computer 
Systems 4 4 8 
Management a _- — 
Ls 34 63 
Dig lice Line EVeCemucal 
Engineering Me 22 
i cee 12 2 14 
ngineering 
Za 20 47 
Total 56 54 110 


fed 


ae 





QPR values were obtained: from the student performance records 
on file at the adaigt were Office, NPS. Both sets of data 
are listed in Appendix C, using a student identification 

code number randomly assigned to each student to preserve 


anonymity. 


B. TEST AND MEASUREMENTS 
To investigate the relationship of English proficiency 
and academic success of foreign students, two variables were 
used in correlational analysis: the set of ECL scores as 
the independent variable, X, and the set of QPR values as 
the dependent variable, Y. These data were tabulated for 
Bitatistical analyses using two packaged sets of computer 
programs: the Statistical Package for the Social Science 
olor 2 | and time JOD Library for the Statistical Pro- 
gram (STATLIB) [Ref. 10]. 
The nature of these two variables will be discussed 
below. 
ee EG Laake Sate 
The ECL test is a primary quality control system 
conducted by the Defense Language Institute at the English 
Language School, Lackland Air Force Base, Texas and at many 
overseas Military Assistance Advisory Group and Missions. 
The ECL is used to sicreen applicants from non- 
English speaking countries for military training in the 
United States. 
In light of the importance of English as a primary 


Communication media for training in the United States, the 


As 


U.S. Department of Defense specifies that a knowledge of 
English language is a prerequisite for foreign students to 
attend schools in the United States and requires that all 
Students must obtain passing scores on the English Compre- 
nensiton Le ve lwhes t . 

An ECL test takes about 60 minutes to administer, 
and tests reading and listening comprehension. The results 
of the test range from zero to one hundred points wnich are 
translated into qualification levels as established in a 
numerical system by DLI. For instance, an individual with 
a 70 ECL knows much more than twice the amount of English 
compared to one with a 35 ECL, and the progressive propor- 
tions between a 45 and 90 ECL are not the same as those 
between 35 and 70 ECL. 

The minimum ECL requirements for entry into various 
schools in the United States are determined by the schools 
of each different service and confirmed by the DLI Head- 
quarters. Various levels of English comprehension are de- 


fined as follows [Ref. 11]: 


Ome 598 ECL : beginning language students 
GU 5 98 ECE ; elementary-intermediate level 
GCM 098 ECE molar reattousrevel|@ior low- Nevel 


Simp Lent tees Speci allized tranming 

i / 9. ECL aictletcatlone level for beginner 
courses where the language limita- 
tion is critical 


80 ECL and above*: qualification level for professional 
career and advanced courses 


*ECL required for entry into NPS belongs to this 
category 


BS 





The Ra elle aaiemeten ace mistics of the ECL tests 
are shown in Appendix A. 
2. Academic Success at NPS 
A student's academic performance at the NPS is 
evaluated on the basis of a quality point number assigned to 


letter grade achieved in a course as follows [Ref. 12]: 


Performance Grade Point Value 

Excellent A 4.0 
A- Sa 
Bt ee 
B 530 
B- 14, WY 
Gr Zs 
C ZO 
C- Le 
D+ Las 
D 1.0 

Failing X 0.0 


When the quarter hours value of a course is multi- 
plied by the quality point number of the student's grade, 

a quality point value for the student's work in that course 
is obtained. 

The sum of the quality points for all courses di- 
vided by the sum of the quarter hour value of all courses 
gives a weighted numerical evaluation of the student's per- 
formance termed the Quality Point Rating (QPR). 

A student achieving a QPR of 3.0 has maintained a 
"BY average invall courses undertaken. 

Academic success at NPS is defined as the successful 
SoOnplCmlonmmomealiecOUrSeS Oreameuxriculum, Supported by a 
final minimum QPR of at least 3.0, and an approved thesis, 
if required. 
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C. ANALYTICAL PROCEDURES 
1. Correlation Analysis 

To investigate the relationship between the two 
variables, ECL and QPR, regression and correlation analyses 
was conducted. A standard statistical technique for evalu- 
abinoeene er fectiveness OfeteSt@scores for predicting aghieve- 
Memes @nescOorrelatwonal validity coerficient. 

Correlation coetficients provide indices of the "ex- 
tent to which excellence on one measure is associated with 
excellence on another measure, or, in other words the de- 
gree to which variables or measures vary together. Mathe- 
Matically, the value of the coefficient may range from a 
Hetmeet DOloltlve comnelaction (11.0), through no relationship 
fOeUjemeto a verter: negative correlation (-1.9). 

tie eencadinmvictmcorrelating’ varirablest, the coeffi - 
cients indicate roughly the following strengths of relation- 


Ships between the variables [Ref. 12] 


besse than ~2Z00---Slight, almost negligible relation- 
ship 

20 - .40 ---Low correlation, definite but slight 
relationship 

~40 - .70 ---Moderate correlation, substantial 
relationship 

ee 90 ---High correlation, marked relationship 

Oe Le. 00 aa Peer TiecOrne latton, Very sdepend -— 


able relationship, 


Howe Vicwmcicmnterpretamioneor the value ofeascorrela- 
tion coefficient is purely relative to the circumstances 


under which it was obtained and should be interpreted in the 


Zo 


light of those circumstances. Many who have employed tests 
for vocational guidance or vocational selection have fol- 
lowed a tradition that the minimum validity coefficient for 
a test to be practically useful is about .45 [Ref. 13]. 

Recent experiences have shown that this standard 
can be too rigid. According to McKenna, it is noted that 
most validity coefficients developed by personnel research 
range from .25 to .50. Yet these validities can have prac- 
tical value, and it is unusual to have a validity coeffi- 
cient of more than .50 [Ref. 14]. 

The correlation coefficients for this study were 
calculated by SPSS using the Pearson product-moment method. 
Statistical results are shown in the tables and figures of 
Chapter IV. 

fee hoeressi10n Analyses 

While the correlation coefficient is a useful index 
for evaluating the correlation between variables, this in- 
Poermiaetton calnot @1VCHadme lear picture of tne pyedicted aca- 
demic achievement of individual foreign students who scored 
aoe Va ousmlevels on thesECL ftestt: 

However, through the use of regression analysis, 
such predictions can be accomplished by establishing a line 
Oum reesression for ECE tesit scores versiis academic grades. 
Titanic diwerunetionvcan be ExXpressed as am albegraic equa- 
PLO P-—-s Aue Whiten provides the cStimate of an unknown 
VariablieyY when the value of another variable X is known 


hRet wy IGF. 
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For each case of analysis, a scatter diagram was 
obtained with the dependent variable plotted on the ordinate 
and the independent variable along the abscissa to check 
for any curvilinear relationships, but none were found. 

The linear line of least squares was computed and plotted 
from the formula Y = a + bX by substituting coefficients "a" 
and "pb", 

For the purpose of summarizing and displaying graph- 
ically the probability or likelihood of individual student's 
academic success, an individual expectancy chart was drawn 
from the scatter diagram. An expectancy chart is a graphic 
display of presenting the probability of attaining success 


associated with a given score range [Ref. 16]. 


Zul, 





ITV. ANALYSIS OF THE RESULTS 


A. ANALYSIS OF DATA 
ieee nc bareLonwor EGE to OPR 

Table V contains a Seer oo summary of ECL and 
QPR. Considering the total sample, the validity coefficient 
of correlation was Found te De 2.2597, Which 1S sipnificant 
acatne. dime level . 

To check for the reliability of the statistical 
results, these results were cross validated by dividing the 
Sanpete so0 Students of the AX~19/Seelass aswal develop- 
mental group and 54 students of the AY-1974 class as a 
validation yroup. The AY-1973 class showed a positive cor- 
Molaguon ome. 5.100, whieh is *simightly higher than the coef- 


Bic leita, OZ 0a ter AY -19747 
Table V. Statistical summary of ECL and QPR. 


AY-1973 AY-1974 Total 


Number of sample 56 54 110 
ECL Mean 86.96 86.33 $0.05 
5 a. Shs 4.94 5.45 
Range 80-100 §0-100 80-100 
QPR Mean Seely oe 3 pees 
Sra lb) 0.40 0.39 Oro 
Range ees See 2-5. 97 2.257 5298 
Morne lation Coe ficient O31 0.26 Oe 
Standard Error of Estimate Oy Sy: tes: Oars 
Confidence Band: Upper S205 oe) 1 Sor 
: Lower 1 TOS, Lalas Pelal, 
RC Ceol OUmn Be LEC e “lal: ee / Sy 1.34 
ReECnes Siomesiepe ('b" OZ 02 Or 02 


Figure 1 shows the estimated regression line and 


associated confidence band’. 
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80 84 88 OZ 96 100 


ECL Scores (X) 


“Regression coefficients a and b were significant 
a ee Ot 


Figure 1. Estimated line of regression based upon 110 
foreign students. 
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ea a a te et 


The regression line was determined by substituting 
computed values of regression coefficients a = 1.3373 and 
b = .0209 in the general equation Y = a + bX, and computed 
values of Y were determined and plotted as a solid line in 
Figure 1. The confidence band in the Figure 1 was estab- 
lished as plus and minus one standard deviation (standard 
error of estimate) and is represented by the dotted lines. 

To show an individual's chances of academic success 
at NPS, the relevant data were tabulated and presented 
graphically as an expectancy chart in Figure 2, and as a 


Pamilation im Table VI. 


EXPECTANCY CHART 


ECL SCORE CHANCES IN 100 OF BEING SUCCESS | 


SERIA OOO OQ 2 





0 25 20 19 100 


onc Expectancy chart for Hmdividual prediction of 
academic success based on 110 foreign students. 
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2. Comparison of Curricula Subgroups 


A comparison of the statistical results of the two 
major subgroups combining similar families of curricula was 
made in an attempt to see if there were any significant dif- 
ferences between engineering and non-engineering curricula. 
Table VII is a tabular summary of the relationship of ECL 


test scores and QPR values of the two different groups. 


Table VII. Comparisons of statistical results of curricula 
subgroups. 

Subgroups by Curricula OR/AS Dept.* Eigambept. 
Sample numbers 63 47 
ECL (X); Mean Si. LooU 85.8723 

5 De 5209/0 5.6554 
QPR(Y); Mean 5.0967 Be ol/4 

oeD. 0.3880 02 3932 
Correlation Coefficient 0.3289 0.2978 
Standard Error of Estimate 0.3694 0.3795 


* OR/AS means Operations Research/Administrative Sciences 
*-—Ene., means Enginieering 

The mean ECL scores of students in OR/AS Department 
nS moe 7c ocUe, WikichalSeS! Wohntly higher than 65.8725 of engineer- 
ieee theteane whithlic thettfinalsyaverape  OFPR obtained by the 
students in OR/AS Department, 3.0967, is lower than the 
ec i/Heonptalned sim enpinecring Curricula. 

The correlation of ECL test scores with QPR values 
in the OR/SA Department was .3289. This is slightly higher 
Ehane the 22978 9in engineering curricula. 

Tniewduspammrtes Metween the results for these two 
groups are also depicted in Figure 3, with the estimated 


regression line of ECL and QPR for the OR/AS Department as 
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REGRESSION LINES 


4.00 





QPR SCORES 
(Y) 


3.50 


80 84 88 5 96 100 


ECL Scores (X) 


Note: Regression coefficients a and b were significant 
at ae. UZ. 


Figure 3. Estimated regression lines of two curricula sub- 
groups (OR/AS Dept. N = 63, ENG Dept. N = 47). 


AYE 





a solid line and that of the Engineering Department as a 
dotted line. 

Individual expectancy of academic success, as shown 
in Figure 4, is higher in the engineering curricula for 


student's whose ECL scores were under 90. 
EXPECTANCY CHART 


CHANCES IN 100 OF BEING SUCCESS* 
100 
100 
61.5 


Giles CORE 


od RAXANANAARERANS 
| 3 Gietie* AES 
| 


9 - 94 L\\QQQAQAYAA 


oOGa0 
85 89 oes 1) 
[Sao 
80 84 54.6 
(esd), 





0 ZS 50 75 100 
AANA OR/AS Department Curricula 


_ = | ENG. Department Curricula 
eeaPR™> 35.00 


Figure 4, Expectancy chart for individual prediction of 


academic success in two curricula subgroups 
(OR/AS Department N = 63, ENG. Department N = 47) 


34 





3. Comparison of Rank Subgroups 


In an attempt to see if there were any significant 
differences in the ECL-QOPR relationship between foreign 
student's military ranks, a comparison of the statistical 
results of the two groups of rank, Lieutenant (LT) and be- 
low versus Lieutenant Commander (LCDR) and above, was con- 


ducted. Table VIII shows the results for the two groups. 


Table VIII. Comparisons of statistical results of rank sub- 
groups. 
Subgroup by Rank LT and Below LCDR and Above 
Sample Numbers 56 54 
ECL(X); Mean SO So) a fly CSS. 
>. UE Aa ab Si o>. 4507 
QPR(Y); Mean oye dalabs Se dhe 0) 
>. De ee 5 Oy 
Correlation Coefficient e250 moos 
Standard Error of Estimate ~4063 Ro Sa 


Those students with higher rank, LCDR and above, had 
a slightly higher mean ECL and mean QPR than those of lower 
rank, LT and below. 
The correlation coefficient of ECL test scores with 
QPR values of the higher rank students is .3398, which is 
higher than the .2336 for the group of lower ranking students. 
Regression lines of ECL and QPR for these two groups 
by rank are shown in Figure 5, with a solid line for LT and 
below, and a dotted line for the LCDR and above group. 
Individual expectancy of academic success in the 
two groups by rank is graphically presented in Figure 6. 
In addition, the probability of academic success in accordance 


with ECL cut-off scores established for selection purposes 


1s shown in Figure 7. gs 





REGRESSION LINES 


4.00 


QPR SCORES 
(Y) 





80 84 88 92 96 100 
ECL SCORES (X) 


Note: Regression coefficients a and b were significant at 
< SOF 


Figure 5. Estimated Regression Lines of Two Rank Subgroups 
(Based upon LT and Below N = 56, LCDR and above, 


N = 54.) 
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EXPECTANCY CHART 


ECL SCORE (CHANCES IN 100 OF BEING suCcEss# IN 100 OF BEING SUCCESS#* 


a 
KAA VAAAAAAAAAAVAAAAAAAAAY 100 


100 


AAAS 
_. ee 





0 25 50 75 100 


ce ee LT and below 
et 


CCDR and above 


* QPR > 3.00 


Figure 6. Expectancy Chart for indivisual Prediction of 
academic success in two rank subgroups. 
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B. EVALUATION OF STATISTICAL RESULTS 

As has been stated previously, the statistical data for 
the present analysis was Pee iene a on the sample of 110 
foreign students. The frequency distribution of the academic 
grades (QPRs) was very close to a normal distribution, with 
a range of socres from 2.25) to 3.97, while that of the test 
scores (ECL) were skewed to the left toward 80, because the 
score of 80 was originally used as cut-off for minimum 
English language proficiency. 

As shown in Table V, the correlation of ECL with QPR 
boretbe@total sample was only .29, which 1s a definite but 
small relationship. 

The standard error of the estimate was computed to be 
.3779, meaning that an estimated academic grade plus and 
NiUlcm—er me WwOULdSecontdims65,2/ percent of the true values. 
Using this confidence level, as shown in Figure 1, we can 
say we are 68 percent confident an individual's QPR will 
fall within the indicated band based upon his ECL test 
ScOlcHmor. In otherwordse the individual has an 84 percent 
probability of achieving a QPR equal to or greater than 
Pidtmeo Rind cared sby tie intersection of the lower confidence 
limit (minus one standard error), and his achieved ECL test 
SicOre wemsliowever, based upon the total sample, the correla- 
tion of ECL test scores with foreign student's QPRs is not 
particularly high. This correlation was probably lowered 
by a restriction of range on the ECL scores. As shown in 
Pig mcm tee an Ene lish proficiency of a foreign student 


PoTemooenGueand above she wolilid have a perfect chance of 


Ng) 





succeeding, or, otherwise, his chances of success would be 
around two out of three. 

Analyses were also made by dividing the total sample 
into two different curricula subgroups: OR/AS and engineer- 
ing. As depicted in Table VII and Figure 3, the comparison 
of the statistical results for these two subgroups reveals 
that English language proficiency is more predictive of 
performance in non-engineering curricula than in an engineer- 
ing curricula. We may, therefore, expect that by using data 
generated for the two different subgroups, English proficiency 
for the student in administrative science type curricula 
should be higher than those in the engineering type curricula 
to achieve the same academic grades. Figure 3 bears out this 
el com 

Analyses were also conducted to compare the statistical 
results of subgroups by dividing the student's military ranks 
into two groups, LT and below and LCDR and above. The re- 
sults of the analysis given in Table VIII shows that those 
foreign students with higher rank had a stronger correlation 
of ECL to QPR than did the lower rank group. This is evalua- 
ted as meaning that if present student assignment procedures 
were continued, students with ranks of LCDR or above would 
Dome clucd tO NCEMNOTeeSNcecessiul ateNPS thian offficers of 
lower ranks. 

mieacd1t10n, Lunther analysis, were intended to check for 
CiiCcC mice SSG Gti reTenierGL elt Olt iSCOneS for) entry 


Pie me WS io eC 1 CULE 7 ye it the cut-off score for 
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Hiei tOomilo ge nore iInsitance, weresset at 80 ECL andlabove, 
asetiets Mace cnespresentstimey, the probability®of academic 
success would be 65.1 percent in OR/AS curricula, 76.6 
pcrecht im Cnigincering curricula, and 7070 percent asea 
whole. 

The chart shows that these probabilities of academic 
success do not change immensely until the ECL cut-off score 
reaches 93 ECL and above, but at the cut-off score of 94 
ECL and above the probability of academic success goes up 
to one hundred percent in both groups. In other words, 
the minimum ECL which would have one hundred percent chance 
of success was found to be 94, and no significant differences 
in success rates were found between scores of 80 and 93. 
PircmNeanlomiioat it theshel cut-orf score for selection of 
NPS candidates was set at 94 ECL and above, there would be 
Hoe academitevecliminat@omeat all. 

ResuUlitsuot this studyegive sone support to the effective- 
HOSsmotmethe Cut-off score of Englash comprehension level as 
measured by ECL and confirms that those who acquire ECL of 
Sm ancdeapovesmore oLtem than not have the proficiency mn 


English needed for success at NPS. 


C. RELATED STUDIES COMPARED 

Ase has been discussed in Chapter W1.C, most situdies of 
English proficiency have shown a relatively low correlation 
Wie che sacademleunertormance Of #Loreilgn students in the 
United States. 

iS ISecimiielagile WX, the probability of academics success 


Gierorelen students at NPS ifs 70 percent, which is comparable 
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to the 78#percenit of the Brown study [Ref. 18] conducted 
army sichools The correlation coefficients of the pre- 
sent study of around .30 is comparable to, or slightly 
higher, than most of the previous studies except the un- 
usually high correlation of .7425 found in the Erchinger 
study [Ref. 19]. At this point, the reader should recall 
that the correlation between ECL and QPR suffered in this 


study probably because of the restriction of range on ECL. 


Table IX. Summary of Analogous Study Results. 


Study By Subjects Test Achievement Correlation 
Chase §& Indiana University 
Stallings N = 526 Lado GPA ~1963~ 
ILS 
Allen University of GPA SAMOS 
Houston N = 90 Lado — 
2200 
Brown Army Schools Judge Pass/ 
N = 1159 by In- Fail 78% 
Struc- 
Mors 
Er omen o¢ 1 Air Force Under- ECL Flying Train- 
graduate Pilot ing Academic .1970 
Training N = 169 Grade »7425 
Pinal» Grade” .1/29 
Present NPS BCE QPR 
Investigator Total N = 110 QPR yeASeS) 
ORGAS N = 63 QPR Bozo 
ENG eA] 2 OS 


The disparity in the predictive power of an English 
language test between verbal and written course work and 
technical instruction found in the Erchinger study, is more 
or less in agreement with the findings of the present study 


revealed in Table IX. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

This study was undertaken to investigate and evaluate 
the relationship of English comprehension levels of foreign 
students with their graduate academic performance at NPS. 

To this end, ECL test scores of rheecerne ea Students 
PHeIned prot to the entry into NBS and their final OPR 
scores were used as variables for regression and correla- 
tion analysis. 

The results obtained from the present study revealed a 
positive correlation of .2897 which is a statistically 
-Significant but marginally useful relationship between the 
ECLs and QPRs. The relationship was found to be slightly 
dPeEercmt FOr engineerine, as compared to non-engineeringe 
curricula. It was stronger for students in the OR/AS cur- 
Pico ade oor those lnmene emoinecering Curricula which 
means that some variables other than English proficiency, 
Seem as pre-knowledge am this field, strongly influence the 
academic performance of students in engineering curricula. 

Piet ata On, themcolpartgommos the OPR-ECI relationship 
within different rank groups showed that foreign students 
with military ranks of LCDR and above obtained higher ECL 
test scores and higher final QPRs than those with ranks of 
LT and below. The relationship of ECL to QPR was also found 
to be higher for foreign students with higher ranks than for 


those with lower ranks. 
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Because the ECL scores were originally cut off at 89 or 
above, the predictive power of ECL as assessed by means of 
the ECL-QPR correlation was constrained by restriction of 
range. 

However, the results of this study showed a 1.0 prob- 
ability of academic success for those students who had re- 
ceived a 94 ECL or above prior to entry into NPS. 

In general, the correlation coefficients which were 
found in the present study were not particularly high, but 
the results have to be judged on the basis of values ordi- 
narily obtained in correlational analysis of this kind 


where restriction of range has operated. 


B. RECOMMENDATIONS FOR FUTURE STUDY 

1. It is recommended that tuture studies investigating 
the relationship of English language problems and the aca- 
demic performance of foreign students in the Naval Post- 
graduate School, group the subjects by curriculum or group 
courses of study according to the characteristics of their 
contents, verbal and quantitative. 

For those who may be interested in finding other pre- 
dictors for the academic success of foreign students such 
as "age," "number of years after baccareaurate study," 
number of years in military service," "grade record and 
source of baccareaurate study," "military academy or civilian 
co llliegem ands ‘preschool experience of graduate level” may 
besusotul@predictors. 

It is also recommended that the Naval Postgraduate School 


or interested students consider administering the Graduate 


Aq 






Record Examination (GRE) test to foreign students before or 
around the time they begin to study at the NPS and examine 
the GRE score as a possible predictor for academic perform- 
ance by the follow-up method. The GRE test scores has been 
verified as an effective predictor for academic success of 
graduate level study by many earlier studies [Ref. 20]. 

If the GRE test scores were a good predictor, it might 
be suggested that the ECL test be replaced by the GRE test 
for the selection of foreign students for graduate study at 


the Naval Postgraduate School. 
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APHENDIX@AR PSYCHOMETRIC *CHARACTERISTICS OF THE ECL TESTS* 


The statistical data of the series 6500 test forms A-F 


are as follows: 


A BB G¢ Bb EE 

Maximum Possible Score 100 100 100 100 100 + £4100 
Mean Score OU omer oO OC mmo. 0 02.5 02.6 
Standard Deviation LOmomenOe O 20.5 2s een ike, oe 5 
Mean Difficulty Index OMe 1. CO Camm 00 0.20 4 O4 
Mean Validity Index Oo ee le 25 ee tO Meo) 
Reliability Index for 

Internal Consistency PO ae is DOW 24 no 7 


(Kuder-Richardson 
formula #21) 


Alternate-form Reli- 
Rodi Ge oectricient epee xXC Saxe DXE  S2xF 
(Pearson Produce - oy 38 (06 TNE te 


Moment Correlation 
formula) 


Ji eoource: ~information on ECL Telsts, DLIEL-D, Defense 
Language Institute, Department of the Army, 
26 August 1974 
*2. Validation Procedure 
olx preliminary forms Gf the ECL tests were constructed 
Witheio0 questions im each form, After presenting them fo 
ecu ihe NAV eond eierorCesSmUCeEnNES Bepresenting 35 dit - 
ferent countries, each form was revised and shortened to a 
100-1tem Gest.) Eachtitem was checked fer content validity, 


difficulty level and discriminating power. The items represent 
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a sample of material found in the American Language Course. 
The 6500 ECL series were correlated with the 6200 ECL series, 
and the correlation coefficients ranged from .93 to .95, in- 


dicating a high correlation between the two series. 
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APPENDIX B: LIST OF SAMPLE ECL AND QPR SCORES 


Student Graduate BiG Toit ad. 

Identification Academic Test Average 

Number Year Curriculum Score QPR 
101 EES O.R. 93 2.60 
102 ‘ 88 52S 
103 a a 95 5255 
104 a ‘ 8 4 aces 
105 e i 92 5.05 
106 ie uy 86 igi | 
107 " i 92 52 
108 . 93 2 On 
109 Bs “ 80 8 ol 
110 mr . 85 2.48 
gh abdl e = 82 2.42 
hike ue ” 81 2.43 
WS es s 84 2.50 
114 = yi 84 2.5 
1s fs General 85 3.60 

Management 
PALLG Y ats 100 50S 
id 7 fy a 94 Poy 
118 ee 89 3.57 
120 ie v 80 3.40 
17 i ie 85 3.18 
122 a 4 80 Bee ai 
IZ us mM 90 See 
124 i Compute mr 89 Saleh 
Management 
Zs . a 84 SD) 
126 7 z 81 Sane 
LeZy, Y e 86 3.24 
128 ue os 94 3.50 
LZ 9 " 87 SA 
201 e Electoigmweas 88 Lees 
Engineering 

202 D t 98 655, a! 
205 _ a 84 a oul 
204 u E 93 Sa 
205 a o 86 S0g 
206 i se 80 Sree 
207 a 92 SL 
208 : _ 80 3.09 
209 . 97 Bi WA! 
210 _ A ii 6) OS 
DUAL . e 88 6) Ab. 
ZA a SI Sees 
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Student Graduate jai On F Total 


Identification Academic Test Average 
Number Year Curriculum Score QPR 
Z15 LLe iS Pile ct rica l 80 2.84 
Engineering 
214 in a 81 2.91 
Bd i ie 84 3.26 
216 ay Mechanical 84 3.49 
Engineering 
Zeley is 95 5. W4 
jogllits _ 80 2.86 
219 oS 80 5555 
220 " os 86 3. 34 
274 “ i 80 3.16 
DL a - 88 a. 24 
Z25 ‘ . 80 S02 
224 om a 93 2.49 
ibs: i = 89 3.49 
226 ‘i ‘ 83 Seer 1 
Zoe " . 80 5.04 
301 Por 74 Ook: 80 2.88 
302 - u 84 3205 
303 - 93 2.90 
304 . 89 3.97 
305 “ 83 oy ie 
306 - 83 2. OW 
307 a u 97 3.43 
308 s - 87 De 
309 : a 99 S07 
310 a os 85 gay 
Sual " A 90 7.97 
312 ut = 81 3.01 
313 ss is 86 SG 
314 7 7 92 2208 
S15 a 80 eo) 
316 . 92 2 
317 ue 94 3.48 
318 ot a 89 1b AS) Ve 
319 “3 - 80 3.09 
320 ay - 89 52) 10 
O41 i 84 2.34 
S22 o ve 82 2.45 
525 ie . 88 sy || 
S74 i ih 8§ Lele 
Sf a General 85 cal 
Management 
326 a - 88 5a 04 
52a . . 88 3.47 
328 is v 89 3.41 
329 " ey 90 S55 
330 - Computer 80 ZO 
Management 
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Student Graduate ECL TO tatl 


Identification Academic Test Average 
Number Year Curriculum Score QPR 
So (ewer Computer 84 53.58 
Management 
552 C 4 85 S25 
333 a ‘ 88 5.20 
334 . 81 3.07 
401 ETeeerica | 88 Sr Gal: 
Engineering 
402 - se 80 Zz 40 
403 . a 89 nee raalt 
404 . di 84 5.29 
40S i" a 93 3.29 
406 ie 87 2.94 
407 o i 88 S05 / 
408 is 84 5.25 
409 o : 80 3.38 
410 " be 80 3.49 
411 a a 84 a 07 
412 - a 87 Soe 
413 as id 88 3.89 
414 i 81 fhe toes 
415 “ Wu 85S 3.49 
416 - . 80 or ak 
417 3 87 Cou 
418 u " 100 Sok 
419 an Mechanical 80 AN 
Engineering 
420 C e 84 3.83 
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